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Using Data: The Math's Not the Hard Part 



In this issue brief, author Craig Jerald highlights 
research collected in the July 2005 special issue of the 
Journal of Education for Students Placed at Risk to 
argue for collecting and using data to increase student 
achievement. 

By Craig D. Jerald 

For many years, experts have counseled school leaders 
to obtain and analyze data as part of their school 
improvement efforts. But a big obstacle stood in 
the way: Essential data were very difficult to collect, 
maintain, and obtain. Some schools made valiant 
attempts to do so, and their efforts often paid off in 
better planning and greater progress. But most schools 
simply did not have the time and knowledge to collect 
or chase down the data they needed. 

Recently, however, those old obstacles have begun to 
crumble. During the last five years, states and districts 
have invested millions of dollars to do a better job 
collecting and storing valuable data. And, just as 



importantly, they are investing still more money to 
provide teachers and administrators with easy-to-use 
websites and software programs offering free and easy 
access to a much wider range of valuable information on 
their students, their classroom, and their schools. 

Better access to data offers an unprecedented 
opportunity for educators to become problem solvers, 
using hard evidence to analyze student performance 
and craft data-driven school improvement plans. But 
information is just a tool, and, like any tool, it is only as 
powerful as the use to which it is put. Many schools are 
not capitalizing on the newly available data at all, and 
still others seem to be missing the point entirely. 

Power and Possibility 

Until recently, technical barriers have prevented the 
wealth of data collected by many school systems from 
being put to practical use. According to Jeffrey Wayman, 
an education data expert at the University of Texas at 
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Austin, "Although schools have been 'data 
rich' for years, they were also 'information 
poor' because the vast amounts of available 
data they had were often stored in ways that 
were inaccessible to most practitioners ." 1 Even 
when data were made available, too often they 
were in a form that made the information hard 
to understand and manipulate. 

But that is rapidly changing. During the last 
decade, education officials have invested 
huge sums of money to develop or purchase 
software that can put information at teachers' 
fingertips in ways that make data — including 
assessment results generated by more 
frequent testing required by No Child Left 
Behind (NCLB) — easier to understand and use 
than ever before. Two thirds of states now 
provide educators with access to interactive 
databases full of school-level information, 
and about half of them offer tools that permit 
teachers to access student assessment results 
over time . 2 Districts are beginning to invest 
heavily as well. According to The Dallas 
Morning News, "Several vendors sold more 
systems [to districts] last year than in all 
previous years combined ." 3 

And that investment seems to have paid 
off. Stories about how schools have begun 
to use these new tools are slowly beginning 
to accumulate, and some offer tantalizing 
glimpses of the positive things that can 
happen when schools use data well. 

Two examples from Boston schools illustrate 
this point. In one school, teachers tested 
a hunch that low scores on standardized 
assessments were probably the result of high 
absenteeism among some students. But the 
analysis showed no relationship between test 
scores and attendance rates. "Reviewing the 
data and eliminating teacher assumptions that 
the problem was only an attendance issue 
allowed for more productive discussions about 
the content and quality of instruction provided 
to students, teacher expectations, and the 
ways in which the [school] might engage 



students more effectively in instruction as well 
as school attendance," say Mary Ann Lachat 
and Stephen Smith, who followed the school's 
progress as part of a case study . 4 

A team from Boston's McCormick Middle 
School analyzed student assessment results 
by classroom to determine whether some 
teachers were doing a better job helping 
students master particular mathematics topics. 
The answer was yes: "The students of one 
teacher excelled on questions dealing with 
graphs; for another teacher, it was fractions; 
for a third, prealgebra topics," say a trio of 
Harvard faculty members who helped lead 
the workshop during which the analysis took 
place. "A consequence of the workshop team's 
findings was that the math teachers decided 
to focus professional development in the 
next year on teaching each other their best 
practices ." 5 

The benefits of data analysis can extend 
well beyond solving a particular student 
learning problem or instituting a new practice. 
Researchers have found evidence that when 
teachers and administrators examine data 
as part of the school improvement process, 
school improvement teams become more 
efficient and effective, decision making 
becomes more collaborative, teachers develop 
more positive attitudes about their own and 
their students' abilities, and educators begin 
to feel more in charge of their own destinies . 6 
Perhaps the biggest benefit is the change in 
professional culture. "We've always based 
solutions on hunches," says Darrell Brown, an 
administrator in Beaumont, California. "No 
more. Now there's a districtwide culture of 
inquiry ." 7 

Missing the Boat 

Unfortunately, along with all of the positive 
examples, there also is evidence that other 
educators are failing to capitalize on the new 
data at their fingertips. 




Some schools are using the data only in 
nominal or symbolic ways to comply with 
requirements. For example, many districts have 
begun requiring schools to analyze data as 
part of the school improvement process. But 
some principals comply with such requirements 
by simply appointing a "data person" who 
knows enough about the software to generate 
a few charts and graphs to include with their 
written plan. Kathryn Parker Boudett and Liane 
Moody describe how conversations about 
"doing the data part" too often conclude: 

"I'm pretty familiar with the software they're 
asking us to use and I think I can run a bunch 
of analyses and get that improvement plan 
drafted before school even starts ." 8 That 
approach often leads to using data to justify 
decisions that have already been made or to 
support, rather than investigate, a set of prior 
assumptions about achievement . 9 

Schools that use data in such nominal or 
symbolic ways might be "complying" with 
expectations that they analyze information as 
part of crafting improvement plans, but they 
are doing so in ways that will not actually make 
those plans better. Such schools fall far short of 
the deeper analysis that can help clearly define 
problems and lead to appropriate solutions, 
and they never develop the "culture of 
inquiry" that is at the heart of the continuous 
improvement process. 

Other schools are finding ways to make more 
instrumental use of data but not necessarily 
for schoolwide improvement planning. 
Researchers conducting training sessions on 
data analysis for educators from one school 
district found that for some teams, "the 
link to instruction was [...] elusive, and the 
intention to 'game the system' seemed to 
take precedence over making constructive 
changes in how students were taught." The 
researchers observe, "This led them to focus 
their attention on students whose scores fell 
just below the minimum passing score and 
to analyze what types of questions these 
students answered incorrectly" so that they 



could provide those students with extra test 
preparation . 10 

A group of researchers studying use of a 
popular data software system in New York 
City public schools observed the same 
phenomenon. "One group of learners 
who often figured prominently in teachers' 
interviews were the students who had scored 
just above or below the edge of proficiency," 
they said. "This population was often referred 
to as the 'bubble kids' because of their 
statistical location in a bubble of scores near 
the cutoff point." Administrators spoke "quite 
frequently" about such students and strongly 
encouraged teachers to use the software 
to identify and provide those students with 
extra help, often in the form of drill and test 
preparation . 11 

Some New York City administrators appear 
to be taking that approach to a questionable 
extreme: 

A deputy superintendent described a district 
policy that had been in place for 1 year 
that he termed "moving test scores": The 
district identifies students who are near the 
proficiency level (at the top of Level 2 but 
just under Level 3) and requires principals to 
target this small group of students, placing 
them with the best teachers and extra 
supports. 12 

Setting aside the ethical implications of such 
policies (should the best teachers really go to 
near-proficient rather than lower performing 
students?), using data analysis software 
primarily to target extra resources to move 
a handful of test scores represents a wasted 
opportunity. A small number of students in 
the school will benefit, but what about next 
year's students — and students the year after 
that? Using data only to target "bubble kids" 
for extra help without digging deeper to 
investigate why student achievement looks 
the way it does will not lead to schoolwide 
improvement. Educators who use data this 
way might help their schools meet NCLB's 
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accountability targets — at least until the 
number of "bubble kids" diminishes as state 
proficiency targets increase — but they will be 
complying with the letter of the law rather than 
its spirit. 

Helping Schools 
"Seize the Data" 

Rather than simply assuming that schools will 
use data well, district leaders and assistance 
providers should begin to work proactively 
to ensure that teachers and administrators 
leverage data for long-term, schoolwide 
improvements. While there is no single formula 
to guarantee that schools will use data well, 
researchers and expert practitioners have 
identified several strategies that can help. 

USE DATA COACHES. Mandating that 
schools use data is insufficient. "We've learned 
that when you do a district directive, you get 
resistance to it," says Darrell Brown of the 
Beaumont Unified School District. "So we 
trained people who want to use it, then when 
they see the value of [the data system], they 
sell it to their colleagues ." 13 Some districts 
have found that training an in-house "data 
coach" can be an effective stepping stone to 
better data use among the rest of the staff. 

PROVIDE BETTER TRAINING. "Training 
that schools provide for their teachers and 
administrators in using data tends to focus 
on how to use software," warn Harvard 
researchers Nancy Sharkey and Richard 
Murnane. "Although this training is necessary, 
our experience has demonstrated that teachers 
and administrators also need to learn a more 
difficult set of skills: how to ask instructionally 
relevant questions of data and how to answer 
such questions ." 14 

ADDRESS FEARS. School culture factors 
might need to be addressed. In conducting 
case studies of data use in urban schools, Mary 
Ann Lachat and Stephen Smith found that, 



"Even when teachers are given training and 
time to think about using data to inform their 
practices, they may be reluctant to do so in a 
culture where they feel threatened or fear they 
will be attacked for something they are doing 
or not doing in the classroom. Effective data 
use requires a culture that is driven by inquiry, 
not fear ." 15 

In a study of three schools that had effectively 
involved entire faculties in data analysis, 
researchers Jeffrey Wayman and Sam 
Stringfield identified a method they call 
"nonthreatening triangulation of data" — 
using multiple assessments of various kinds 
throughout the year to gauge student progress 
rather than relying only on scores from annual 
standardized tests . 16 

District leaders and assistance providers 
also can help teachers overcome their fear 
by working with principals to establish an 
open, nonthreatening climate for examining 
data. Some experts also recommend using 
established questions and formal processes to 
frame data discussion until teachers become 
comfortable enough to pursue open-ended 
conversations. 

DEMONSTRATE LEADERSHIP. Strong on- 
site leadership is another key to successful 
data use. Wayman and Stringfield found, 
for example, that "teachers in each school 
explicitly singled out their principals as a major 
factor in the success of their data initiatives ." 17 

On the other hand, principals who are 
not enthusiastic about using data to solve 
problems themselves can dampen the 
enthusiasm of teachers and even prevent 
good analyses from having a positive impact 
on practice. Researchers have found that even 
when a data team embraced data analysis and 
promoted data-driven instruction, "Lack of 
support from school leadership meant that the 
team's work was not likely to make a difference 
in school practices ." 18 




PROVIDE SUFFICIENT TIME. Even with good 
training and strong leadership, teachers need 
enough time to learn to use data thoughtfully 
and additional time to practice what they 
learn. Ethan Minz, Sarah Fiarman, and Tom 
Buffett found that, "Pressed to solve significant 
problems quickly, many educators take the 
swift route: look at a table or two, make a 
judgment based on what they already think 
is true, and decide to address a problem that 
they can solve easily and that doesn't require 
much change on their part." Time constraints 
can create a "stuck point," where teachers 
become frustrated and blame students 
when assessment results do not improve, 19 
but solving the problem of too little time is 
complex and will require creative solutions. 

Wayman and Stringfield describe several such 
solutions employed by principals of the three 
schools they studied: 

While noting the difficulty of finding time to 
give teachers for data use, administrators 
and principals also voiced the importance of 
finding ways to do it. Principal C described a 
system for creatively working the contractual 
planning time into her data initiative. Principal 
B was infusing data use into school teaming 
structures and sometimes used faculty 
meeting time to explain data methods. 

Principal A used a variety of meeting 
structures already present in the school to use 
data for learning decisions. All three principals 
used staff development days to help teachers 
learn methods and examine student data . 20 



MODEL. Fi nally, districts and assistance 
providers should look for opportunities 
wherever they can find them to model 
thoughtful data use. Educators often need to 
see this new way of working in action in order 
to understand it fully — and also to believe 
that district administrators are serious about it 
themselves. School improvement teams that 
are just beginning to use data should be given 
the opportunity to observe how teachers use 
data in schools that are ahead of the curve. 
And districts should establish their own data 
teams that allow system administrators to 
publicly model how to identify educational 
problems and solutions using data. 21 

Conclusion 

American education stands at the brink of its 
own information revolution. But the revolution 
will not be automatic. Educators will need 
plenty of support, not just to decipher the 
statistics and operate the software but to link 
data to instructional problems and solutions. 
How much help they get will determine 
whether the new tools get used simply to push 
a few test scores over the proficiency line or to 
enable whole schools to establish the "culture 
of inquiry" that forms the cornerstone of 
continuous improvement. 
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This is the fifth in a series of issue briefs to be written for The Center for Comprehensive 
School Reform and Improvement during 2006. These commentaries are meant to help 
readers think beyond simple compliance with federal law or basic implementation of 
programs: What unacknowledged challenges must educators and leaders confront to help 
schools operate more effectively and to sustain improvement over the long run? In what 
ways does the conventional wisdom about teaching, learning, and school improvement run 
counter to current research and get in the way of making good decisions? What are the 
emerging next-generation issues that educators will face next year and five years from now? 
Readers can visit www.centerforcsri.org to obtain other papers in this series and to access 
additional information on school reform and improvement. 
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